1. Field of the Invention
This invention relates generally to the storage, archiving, networking, and retrieval of medical images and video, and more particularly to improvements in a Picture Archiving and Communications System (PACS) operating in networks having different possible bandwidths.
2. Description of the Related Arts
A PACS is a system for the storage, retrieval, and display of medical images. A PACS typically consists of one or more networked computers along with a substantial amount of semi-permanent digital storage in the form of for instance, a RAID (redundant array of inexpensive hard disks), tape storage, or optical disks. A PACS also typically includes software for storing, retrieving, and displaying images, along with hardware that may be necessary for physical management of digital media (e.g., a robotic tape loader), display, and input.
A PACS is typically connected to an imaging device such as a CT (computerized tomography) scanner, an MRI (magnetic resonance imaging) scanner, or an X-ray machine capable of providing images in digital format, often including images compliant with the DICOM (Digital Imaging and Communications in Medicine) format. A doctor or other health care provider uses the imaging device to create a digital picture of a patient for diagnosis or treatment purposes. The image is delivered via a network to the PACS, where it is stored along with information identifying the particular patient. The image is viewed on the PACS immediately or it is retrieved for display later. The image is optionally processed prior to storage, or it is stored in a raw digital format and subjected to optional processing later.
Prior to the development of PACS technology, hospitals typically stored medical images on film that had to be catalogued and retrieved by hand. Early computerized medical imaging devices were flawed because the machines were typically standalone devices with no or limited archival capabilities and proprietary file formats. PACS, along with the standard DICOM and other file formats, provided a convenient, standardized way to store medical images with fast, electronic retrieval, more convenient backup, and potential for remote electronic distribution.
Despite their advantages, traditional PACSs have numerous shortcomings. First, a traditional PACS may operate in connection with various different networks, each of which has a different bandwidth, or even with a single network having an effective bandwidth that varies with overhead requirements, competing data traffic from other systems using the same network, and the like.
Known PACSs are configured to stream images using any of a number of data transfer protocols. Some protocols are particularly well adapted for high bandwidth networks, while others typically are optimized to work with lower bandwidth networks. Commonly, optimizing a protocol for a particular network bandwidth calls for certain design tradeoffs. For example, high bandwidth channels permit the transfer of image data with very little preprocessing or post processing, thus imposing very little overhead on the devices used to store, retrieve, package, transmit, receive, and decode the data. Other communication channels, for example wireless channels, may have a very wide bandwidth but suffer from periodic network failures, for example when a device's wireless communications path is blocked by other equipment or where the device gets out of range of the node it is communicating with. Still other channels, such as networks relying on conventional modem communications over telephone networks, particularly older POTS (“Plain old telephone service”) networks, have far less bandwidth. To achieve satisfactory image streaming using such low-bandwidth networks, significant data compression/decompression must be used, typically calling for significant processing resources.
In many instances, it may be acceptable to choose a likely network bandwidth for a PACS implementation, and select communication protocols that are optimized to work with such expected network. However, experience has shown that quite often such systems will end up operating under various network bandwidths, and it would be advantageous to have a PACS that could operate effectively in connection with a variety of network bandwidths.
Considering the situation in greater detail, medical images shared across one or more healthcare organizations are commonly stored in the DICOM format, which supports a variety of data representations, including raw pixel data, baseline lossless compression (lossless JPEG), and the progressive compression standard known as JPEG 2000. Healthcare organization network infrastructures often suffer from bottlenecks, insufficiencies, instabilities and other problems that make image streaming using a preferred format impossible, for example because the images cannot be fetched and rendered at a user's viewing workstation fast enough to appear in real time.
Simple known solutions, such as increasing network bandwidth, are not always feasible, due to a variety of factors ranging from cost to hospital wiring policies.
Another possibility is conventional compression of the image data to be streamed. In many medical applications, however, some types of compression, particularly lossy compression, are either disfavored or outright forbidden. The concern is that medical diagnostic work is too important and subtle a task to be burdened with additional uncertainties that may arise by intentionally degrading an image from its original number of pixels, bit depth, and other characteristics. Lossless compression techniques, even when they are allowed, typically only reduce image file sizes by less than an order of magnitude, which in many cases is not sufficient to provide real time streaming over low bandwidth channels. Medical images are particularly ill-suited to lossless compression because many medical imaging modalities such as CT or ultrasound inherently produce images having significant pure noise components that do not lend themselves to significant compression using known lossless methods.
Even where compression is workable, difficulties remain to be addressed. For example, the wavelet-based JPEG2000 image compression standard provides progressive compression capabilities that allow the transmission of images in separate “layers” of quality. An initially transmitted image improves with quality as more data arrive. In “lossless” mode, JPEG2000 processing allows transmission of the data until the received image is of the same quality as the original image. In some circumstances, use of this format provides some manner of inherent adaptability to varying bandwidths, as once bandwidth drops below the size needed for real time lossless transmission, transmission simply continues at somewhat lower quality, with images being discarded before they reach original-image quality.
A further known improvement using JPEG2000 is to transmit high quality images for only a particular region of interest (ROI) when full-bandwidth original-quality transmission is not possible.
These solutions may be available where original images are available in a DICOM format compliant with JPEG2000, but many imaging modalities do not make images available in such format. Even where such images are available, lossless JPEG2000 coding is computationally demanding, typically 5-6 times slower than comparable lossless coding under the previous JPEG standard. Correspondingly, decoding and rendering of such JPEG2000 images is likewise more computationally intensive, and may result in unacceptable performance on conventional viewing workstations.
U.S. Pat. No. 6,314,452 discloses one partial solution to the difficulties in using JPEG2000 using wavelet streaming. However, this and others of the known techniques still impose unnecessary overhead, for instance by still coding/decoding images when wide network bandwidths are available and image transfer could be accomplished without this additional processing.
Some on-demand video applications also address similar issues through scalable video streaming and quality of service. Using the advanced capabilities of the MPEG video compression standard, one can control the rate/distortion tradeoff on a per-device (i.e., viewing device) basis. Accordingly, one can dynamically allocate a bitstream among various viewing nodes and stream higher resolution video to higher resolution viewing devices and higher visual quality to certain users. However, these solutions typically assume that video is streamed from a single pre-processed MPEG file composed of resolution and quality layers—an assumption that is often not true in many medical applications.
In medical tele-radiology applications, the traditional solution has been to schedule transfers of large lossless DICOM images in advance, during off-peak hours, so that a radiologist or other provider can receive, store, and later review the images. Various techniques are used to try to predict when off-peak slots will be available and match that with the healthcare needs associated with the images to be transmitted.
U.S. Pat. No. 6,848,004 discloses a system for adaptive delivery of rich-media content in web pages. This document discloses a client-server system in which the client calculates the bandwidth and the server adaptively transmits web content, with richer content being transmitted when higher bandwidth is available. This technique relies on the availability of dynamically loading applet technologies, such as JAVA applets, that may not be available in all situations.
Likewise, U.S. Pat. No. 6,243,761 discloses a method for dynamically adjusting multimedia content of a web page by a server according to effective bandwidth and/or latency characteristics. In this instance, the content sent to the client computer is adjusted, such as by reducing the size, resolution, or number of images of the graphic image. While such modification of content might be acceptable in many web-browser applications, it may not be appropriate for many medical applications.
Another approach is described in Chandra, S. et al., Application-level differentiated multimedia Web services using quality aware transcoding, 18 IEEE Journal on Communications 12, December 2000, pp. 2544-2565, ISSN 0733-8716 (“Chandra”). Chandra discloses use of application-specific characteristics of Web services to manage resources. Again, the environments of typical PACSs may not make such services available. Still other disclosures addressing some of these issues are Gaddah et al., image transcoding proxy for mobile Internet access, Proceedings of the Vehicular Technology Conference 2002, September 2002, pp. 807-811, ISSN 1090-3038; Lee et al., SIQuA: server-aware image quality adaptation for optimizing server latency and capacity in wireless image data services [mobile radio], Vehicular Technology Conference, 2004, 4 VTC2004-Fall (IEEE), pp. 2611-2615, ISSN: 1090-3038; Kim et al, A new resource allocation scheme based on a PSNR criterion for wireless video transmission to stationary receivers over Gaussian channels, IEEE Transactions on Wireless Communications 1:3, July 2002, pp. 393-401, ISSN: 1536-1276; Raman et al. ITP: an image transport protocol for the Internet, IEEE/ACM Transactions on Networking 10:3, June 2002, pp. 297-307, ISSN 1063-6692; U.S. Pat. Nos. 5,931,904 and 5,276,898.
Generalizing from the above, there remains a need for a system that uses advanced processing to stream medical image data at acceptable quality when available bandwidth is low, and that avoids doing such processing when it is not needed, for example when available bandwidth is high.